Skip to main content
Version: DAI 7.4

Test Results

In Controller > Test Results you can view the test results for all your models and scripts to make sure that your tests are running correctly. In addition, you can:

  • Determine which of your tests have passed, failed, or are in progress.
  • View the execution steps, logs, and screenshots for the selected test run.
  • See which execution environment was used for each test run.
  • See which system under test (SUT) was used for each test run.
  • Delete test results that are no longer required.
  • Abort tests that are in progress.

Test Results showing all Run Statuses

Filtering​

By default, you see all your test results on the Test Results page. You can limit the results you see by applying filters. DAI provides several types of filters, which are described below.

The top of the Test Results page provides the following options:

  • Search by ID: a search field where you can enter the exact ID of a test result you want to see. The results for each test execution have a unique ID generated by the Eggplant DAI database.
  • Filters: a button you can click to show or hide the filter selection bar. The Filters button also shows the number of filters applied, if any.
  • No filters applied or Clear all button depending on whether any filters are applied.

You might also see the filter selectors, which display in a bar below the Filters button when they are showing. Click the Filters button to show or hide the filter selection bar. For example, screen below shows the Suites filter in use with the selected suite name. You also see the Filters button showing the number 1 in purple, indicating one filter is applied.

Test Results filtered by a Suite

The filter types are as follows:

By ID​

Narrow the results shown to a specfic test result by entering the exact test result ID in the Search by ID field at the top of the Test Results page. The results for each test execution have a unique ID generated by the Eggplant DAI database.

note

You only see the following filter options when the filter selection bar is showing. Click the Filters button to see the filter selection bar if it is not showing.

By Run Status​

Narrow the results shown by the run status. The run status options are as follows:

  • Passed: successful tests are highlighted in green.
  • Failed: unsuccessful tests are highlighted in red.
  • In Progress: incomplete tests are highlighted in yellow.
  • Cancelled: tests that were stopped before they could be completed.
  • Error: tests that resulted in error as well as tests that were interrupted due to loss of network connectivity or the agent process ending abruptly.
note

The All Run Statuses filter option was removed in DAI 7.2. All of the run statuses display when no filters are applied.

By Date range​

Narrow the results shown to a specific timeframe that matters most to you. Click Date Range to see the date filter:

  • After: filters the test results to only those that exist after the selected date.
  • Before: filters the test results to only those that exist before the selected date.
  • Between: allows you to specify a range by selecting two dates.
  • None: removes the date filter.

By Model​

Narrow the results shown to a specific model by clicking the Models filter button and selecting the model you want from the list. For more information about models, see DAI Build a Model.

Enable Show submodels results at the bottom of the Model filter list to show the test results for the main model and submodels, or disable it to show only the main model results.

note

When you filter results By Model, you select the name of a model and then see the results for only that model. If you enable Show submodel results, you will not see the model's submodel results unless the submodel name matches the name of the model you filtered on.

By Suite​

Narrow the results shown to a specific suite by clicking the Suites filter button and selecting the suite you want from the list. Suites contain the SenseTalk snippets (or scripts) that you use in your DAI models or test cases.

note

Script-based test results generated before upgrading to DAI 6.4 will not appear when filtering by suite.

By Test Case​

Narrow the results shown to a specific test case by clicking the Test case filter button and selecting the test case you want from the list. For more information about test cases, see Managing Test Cases.

note

If there are multiple test cases with the same name, the drop-down selector groups them into one selection, but it will be able to filter the results accordingly.

By Test Config / Live run​

Narrow the results shown to a specific test configuration ("test config") or filter the results to show only test configs or only live runs by clicking the Test config / Live run button and selecting the options from the list. For more information about test configs, see Working with Test Configurations.

Test config / live run combo box

By Test Config Tag​

Narrow the results shown to a specific test configuration ("test config") tag. For more information about test configuration tags, see Working with Test Configurations.

By Execution Environment​

Narrow the results shown to a specific execution environment. For more information about execution environments, see Environments.

By SUT​

Narrow the results shown to a specific system under test (SUT). For more information about SUTs, see Managing SUTs in Eggplant DAI.

note

When you change the name of an existing SUT, DAI retains the ID of that SUT. If you filter by the new SUT name, you may see the original SUT name in the filtered results.

Clear All Filters​

You can clear all filters at once by clicking the Clear all button, which is to the right of the Filters button at the top of the Test Results page. Once you click the Clear all button, you see No filters applied in its place.

note

If no filters are applied, you do not see the Clear all button. You see no filters applied in its place.

Results​

The list of test results provides summary information and execution metrics for the results that match your criteria:

  • Status & ID: The status and ID show in the same column.

    • status: The outcome of the test run. You can filter the results by Cancelled, Error, Failed, In Progress, or Passed statuses. Click the status to view the execution details for the test.
    • ID: The unique ID of the test result.
    note

    The Status & ID column sorts by run status, not ID. Prior to DAI 7.2, the Status and ID were separate columns.

  • Test case / Exploratory run: Type of test execution, either: Test case or Exploratory, Replay (a test case name for directed test cases), or a Script name in the case of a script-based exection. Exploratory or Replay display as a label.

  • Model / Suite: Model or suite used in the test run.

  • Test config / Live run: Test configuration to which the test belongs, or Live run. If the test config contains any tags, they display underneath.

note

Prior to DAI 7.2, this column's name was Run mode.

  • Start date: Date and time when the test started running.
  • Duration: The duration of the test runs, formatted in hours, minutes, and seconds.
note

If you export test results to a csv file, the Duration column is formatted in seconds in the file.

  • Environment: The name of the execution environment used for the test run, or None if no execution environment was used.
  • SUT: The system under test (SUT) used in the test run.
  • Failed at: The last failed action for each failed test run.

If a test is in In Progress status, a stop button (stop button) appears in the row. You can stop a test run or test config runs using the button.

note

When you abort a test run that is part of a test configuration schedule, all runs that are part of that configuration schedule are aborted.

  • Actions menu Actions menu icon: Provides the option to delete test runs. You can remove unwanted results, such as failed test results that remain in an errored state.

  • Delete: If the test result was for a Live Run, and not part of a test configuration run. This option deletes just the selected test run.

  • Batch Delete: If the test result was part of a test configuration run. This option will delete the results for all tests in that run of the test configuration. For more information, see Batch Delete below.

note

A test result is considered part of a test configuration if it has a value in its row under the Test Configuration column.

For more information, see Batch Delete below.

Sorting Results​

You can sort results by clicking on a column header on the Test Results page. The columns you can sort show the sort button Sort button. Clicking the sort button the first time, sorts the results in ascending order. Clicking it again, sorts it in descending order. Then it works as toggle sorting in ascending or descending order as you click it.

  • If a column is sorted in ascending order, you see an up arrow ascending sort.
  • if a column is sorted in descending order, you see a down arrow descending sort.

Exporting Results​

For reference or analyzing test run outcomes, you can export your test results as a comma-separated value (.csv) file.

Click the Export button in the top right corner of the Test Results page to export the list of results currently showing on the page. This means the export contains results limited by any applied filters, or all of the results if no filters are applied.

Batch Deletion from Test Results​

Sometimes, test results are generated from a Test Configuration run. Therefore, they have to be deleted in a batch. A test result is considered a part of Test Configuration if it has a value in its row under the Test Configuration column.

To delete the test result, click the Actions menu Actions menu icon and click Batch Delete.

The Batch Delete option will not be available if the test result has In Progress status.

You see a confirmation dialog box informing you that the test result belongs to a Test Configuration run. Click View Batch on the dialog box. This opens the Results page showing all the individual test runs for the Test Configuration run.

At the top right corner of the screen is a red button labeled as Delete Run. Clicking it displays the confirmation dialog box. Click Confirm to proceed with the deletion.

This action deletes the instance of the selected test configuration run and all of its test results.

After the deletion, you see the Test Results page again.

Scheduling DAI to Delete Test Results​

You can configure DAI to delete test results, including logs and screenshots, for you on a scheduled basis by setting the following two environment variables in your DAI Server configuration file.

  • DELETE_RESULTS_AFTER_DAYS

  • DELETE_RESULTS_TRIGGER_TIME

The DAI Server configuration file does not include these variables by default. You will need to manually add them to your file to use this feature. See Enabling Scheduled Test Artifact Deletion for explanations of these variables and instructions for modifying your DAI Server configuration file.

Last updated on